This invention generally relates to performing computations using a matrix processor architecture and more specifically to dynamically managing numerical representation of values of a tensor, for example, in a distributed matrix processor architecture used for implementing artificial neural networks.
Distributed and single processor architectures are used for efficient processing of computations, for example, matrix computations. A distributed processor architecture may be used for implementing artificial neural networks. Artificial neural networks are used to solve tasks that are difficult to solve using traditional computational models. For example, an artificial neural network can be trained to perform pattern recognition tasks that would be extremely difficult to implement using other traditional programming paradigms. Several tasks performed using distributed processor architectures such as artificial neural networks require tensor (or matrix) computations. In order to maintain a high level of precision when performing tensor computations, conventional architectures use floating point number representations for storing values of the tensors. These techniques for representing and performing operations using floating point numbers require significant hardware resources.